Issue Analysis for Residual Structural Coverage in Dynamic Symbolic Execution

The process of achieving high structural coverage of the program under test can be automated using Dynamic Symbolic Execution (DSE), which generates test inputs to iteratively explore paths of the program under test. When applied on real-world applications, DSE faces various challenges in generating test inputs to achieve high structural coverage. Among issues related to these challenges, our preliminary study identified two main types of issues: (1) object-creation issues (OCI), where DSE fails to generate method-call sequences to produce desirable object states; (2) external-method-call issues (EMCI), where symbolic values are passed as arguments to third-party library methods that are not instrumented by DSE. Automatically solving these two main types of issues is challenging, since the exploration space of generating method-call sequences for desirable object states is usually too huge, and instrumenting all third-party libraries can cause explosion of the exploration space. However, when provided with informative information of issues, users can effectively assist DSE to achieve high structural coverage. In this paper, we propose a general approach, called Covana, to identify issues faced by DSE via analyzing runtime information, and filter out irrelevant issues using residual structural coverage. We provide two techniques to instantiate our general approach to identify OCIs and EMCIs. To show the effectiveness of Covana, we conduct evaluations on two open source projects. Our results show that Covana effectively identifies 155 OCIs, and 43 EMCIs. Moreover, Covana effectively reduces 296 irrelevant issues out of 451 OCIs and 1567 irrelevant issues out of 1610 EMCIs produced by a straightforward approach.